Global optimisation of neural network models via sequential sampling-importance resampling
نویسندگان
چکیده
We propose a novel strategy for training neural networks using sequential Monte Carlo algorithms. This global optimisation strategy allows us to learn the probability distribution of the network weights in a sequential framework. It is well suited to applications involving on-line, nonlinear or non-stationary signal processing. We show how the new algorithms can outperform extended Kalman lter (EKF) training.
منابع مشابه
Global Optimisation of Neural Network Models via Sequential Sampling
We propose a novel strategy for training neural networks using sequential sampling-importance resampling algorithms. This global optimisation strategy allows us to learn the probability distribution of the network weights in a sequential framework. It is well suited to applications involving on-line, nonlinear, non-Gaussian or non-stationary signal processing.
متن کاملHybrid sequential Monte Carlo/Kalman methods to train neural networks in non-stationary environments
In this paper, we propose a novel sequential algorithm for training neural networks in non-stationary environments. The approach is based on a Monte Carlo method known as the sampling-importance resampling simulation algorithm. We derive our algorithm using a Bayesian framework, which allows us to learn the probability density functions of the network weights and outputs. Consequently, it is po...
متن کاملSequential Monte Carlo Methods to Train Neural Network Models
We discuss a novel strategy for training neural networks using sequential Monte Carlo algorithms and propose a new hybrid gradient descent sampling importance resampling algorithm (HySIR). In terms of computational time and accuracy, the hybrid SIR is a clear improvement over conventional sequential Monte Carlo techniques. The new algorithm may be viewed as a global optimization strategy that a...
متن کاملSequential Importance Sampling Based on a Committee of Artificial Neural Networks for Posterior Health Condition Estimation
The output of real-time diagnostic systems based on the interpretation of signals from a sensor network is often affected by very large uncertainties if compared with local nondestructive testing methods. Sequential Importance Resampling (SIR) is used in this study to filter the output distribution from a committee of Artificial Neural Networks. The methodology is applied to a helicopter panel ...
متن کاملResampling: An improvement of importance sampling in varying population size models.
Sequential importance sampling algorithms have been defined to estimate likelihoods in models of ancestral population processes. However, these algorithms are based on features of the models with constant population size, and become inefficient when the population size varies in time, making likelihood-based inferences difficult in many demographic situations. In this work, we modify a previous...
متن کامل